CatchIt, a Development Environment for Transparent Usability Testing

No existing tool addresses both the development and evaluation process for interactive systems. Current evaluation tools support either the predictive or the experimental approaches to usability testing, and automated tools that make possible experimental evaluation impose a tricky manual instrumentation of the existing code. In this paper we describe CatchIt, a development and usability testing environment, to respond to these limitations using work domain descriptions as the central foundation. Within CatchIt, work domain descriptions define a reference model that serves two complementary purposes: the model can be used as reusable code from which a new application can be built or it can be used as a standard against which new applications can be tested. Evaluation, whether it be predictive or experimental, consists of the automated detection of deviations of the new application from the norm defined by the reference model. The reference model evolves over time based on the accumulation of past experience. As such, it supports traceability and evolutivity of the corresponding work domain.